1. Field of the Invention
The invention relates to a method of processing images for constructing a target image from adjacent images having a fixed frame line and referred to as source images, said source and target images having substantially common view points.
The invention also relates to an image processing device comprising:
a system of n fixed real cameras arranged in such a way that their individual fields of view merge so as to form a single wide-angle field of view for observation of a panoramic scene, PA1 an image construction system simulating a mobile, virtual camera continuously scanning the panoramic scene so as to form a sub-image referred to as target image corresponding to an arbitrary section of the wide-angle field of view and constructed from adjacent source images furnished by the n real cameras, said virtual camera having a view point which is common with or close to that of the real cameras. PA1 digitizing the images, PA1 determining, for one of the pixels of the target image, the address of a corresponding point in one of all source images, PA1 determining the luminance value at this corresponding point, PA1 assigning the luminance value of this corresponding pixel to the initial pixel in the target image, PA1 repeating these steps for each pixel of the target image. PA1 a system of n fixed real cameras arranged in such a way that their individual fields of view merge so as to form a single wide-angle field of view for observation of a panoramic scene, PA1 an image construction system simulating a mobile, virtual camera continuously scanning the panoramic scene so as to form a sub-image referred to as target image corresponding to an arbitrary section of the wide-angle field of view and constructed from adjacent source images furnished by the n real cameras, said virtual camera having a view point which is common with or close to that of the real cameras, characterized in that this image processing device is a digital device and in that the system (100) for constructing the target image Io includes: PA1 an address computer for causing a point at an address in one of the source images to correspond to a pixel address in the target image, PA1 means for computing the luminance value of the point at the address found in the source image and for assigning this luminance value to the initial pixel at the address in the target image. PA1 either a sequential image-by-image read-out of partitions of the observed scene, possibly with a zoom effect, PA1 or a continuous read-out by scanning the scene observed with the sight and azimuth effect or with rotation. PA1 first means for storing the parameters relating to the virtual camera for supplying the address computer with the scale factor and the orientation of the optical axis of the virtual camera in a fixed orthonormal landmark which is independent of the cameras, i.e. the azimuth angle, the angle of sight and the angle of rotation; PA1 second means for storing the parameters relating to the real cameras for supplying the address computer with the scale factor and the orientation of the optical axis of each real camera, i.e. their azimuth angle, the angle of sight and the angle of rotation in said fixed landmark; PA1 an address generator for generating, pixel by pixel, the addresses (Ao) of the pixels of the target image so as to cover the entire target image, the address computer determining the particular source image and the point at the address (Aq) in this source image, which corresponds to each pixel of the target image, on the basis of the parameters of the virtual camera and the real cameras. PA1 first means for constructing a model (MCo) of the virtual camera with a projection via the view point, PA1 second means for constructing models (MC1-MCn) of the real cameras with a projection via the view point and with corrections of distortions and perspective faults. PA1 first means for computing the geometrical transform for applying a geometrical transform referred to as inverse "perspective transform" (H.sub.o.sup.-4) to each pixel at an address (Ao) of the image of the virtual camera, in which transform the model (MCo) of the virtual camera provided by the first construction means and the parameters for the azimuth angle, the angle of sight, the angle of rotation and the scale factor of this virtual camera provided by the first storage means are taken into account for determining, on the basis of this inverse perspective transform (H.sub.o.sup.-4), the positioning in said landmark of the light ray passing through this pixel and the view point, PA1 means for storing the position of the light ray obtained by the inverse perspective transform (H.sub.o.sup.-4), PA1 means for selecting the particular source image traversed by this light ray, PA1 second means for computing the geometrical transform for applying a geometrical transform referred to as "direct perspective transform" (H1-Hn) to this light ray in said landmark, in which transform the models of the real cameras provided by the second construction means, the parameters for the azimuth angle, the angle of sight, the angle of rotation and the scale factor of the corresponding real camera provided by the second storage means are taken into account, PA1 and storage means for supplying, on the basis of this direct perspective transform (H1-Hn), the address (Aq) in the particular source image which corresponds to the light ray and thus to the pixel of the address (Ao) in the target image. PA1 an interpolator for computing a most probable value of a luminance function (F) at the address (Aq) found by the address computer in the source image furnished by the selection means; PA1 third storage means for assigning the luminance value (F) corresponding to the point at the address (Aq) found in the source image to the initial pixel in the target image at the address (Ao) furnished by the address generator, and in that the system for reconstructing the target image also comprises: PA1 an interface for enabling a user to define the parameters of the virtual camera, which parameters include the scale factor and the orientation of the optical axis.
The invention is used in the field of telemonitoring or in the field of television where shots covering large fields are necessary, for example when recording sports events. The invention is also used in the field of automobile construction for realizing peripheral and panoramic rear-view means without a blind angle.
2. Description of the Related Art
An image processing device is known from Patent Application WO 92-14341, corresponding to U.S. Pat. No. 5,187,571. This document describes an image processing system for television. This device comprises a transmitter station including a plurality of fixed cameras arranged adjacent to each other so that their fields of view merge and form a wide-angle field of view. This system also comprises a processing station including means for generating a composite video signal of the overall image corresponding to the wide-angle field of view, and means for selecting a sub-image from this composite image. This system also comprises means, such as a monitor, for displaying this sub-image. This sub-image corresponds to a field of view having an angle which is smaller than that of the composite image and is referred to as sub-section of the wide-angle field of view.
This image processing device is solely suitable for conventional television systems in which the image is formed line by line by means of a scanning beam.
The processing station enables a user to select the sub-section of the wide-angle field of view. The corresponding sub-image has the same dimension as the image furnished by an individual camera. The user selects this sub-image by varying the starting point of the scan with respect to the composite image corresponding to the wide-angle field of view. The wide-angle field of view has an axis which is parallel to the video scan, with the result that the starting point for the video scan of the sub-image may be displaced arbitrarily and continuously parallel to this axis.
The angle of the field of view to which the sub-image corresponds may be smaller than that of a real camera. However, the localization of the sub-image does not include a displacement perpendicular to the scan; its localization only includes displacements parallel to this scan. The formation of the sub-image does not include the zoom effect with respect to the composite image, i.e. the focal change of the sub-image with respect to the focal length of the image pick-up cameras.
The image processing station thus comprises means for constructing the selected video sub-image line after line. These means essentially include a circuit for controlling the synchronization of the video signals from the different cameras.